The present invention relates to accelerated compression requests and, more specifically, a process for mixing software based compression requests with hardware accelerated requests for a single file.
There is an industry wide direction to introduce specialized hardware accelerators for central processing unit (CPU) intensive functions, some of which include data compression operations. Devices of this nature are more valuable if they can be transparently integrated into existing workloads and applications.
In order to provide for transparent integration of hardware accelerators, some issues need to be addressed. First is that the hardware accelerators will have different latency and speed attributes for compression than performances of compression operations in pure software. One of these speed attributes could be the overhead in communicating with the device. That is, due to this potential overhead there will be a minimum size of data which will need to be provided in order to amortize that overhead. Ideally any software package would provide large amounts of input per request. There are conditions however, due to protocol or data format standards, where large requests will be intermixed with very small requests that typically carry metadata information about the payload. In this environment, use of hardware accelerators for both large and small requests will impact the overall performance of a compression operation of a single file.
The zlib open source library provides the standard programming interface for using the DEFLATE compression file format. The IBM zEnterprise Data Compression (zEDC) support extended the zlib library to use either its existing software interfaces or the new zEDC hardware to perform compression. Today, the determination to use either the software or hardware compression is made on a per-file basis based on the size of the first request for that file. This method has two shortfalls. The first of these shortfalls is that the request may be very small but may be followed by many large requests. The second shortfall is that the first request may be very large and followed by or intermixed with many very small requests. In both of these cases the existing support cannot use the hardware acceleration to achieve the best possible throughput for compressing the file.